Papers by Pranav Narayanan Venkit
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
An Audit on the Perspectives and Challenges of Hallucinations in NLP (2024.emnlp-main)
Copied to clipboard
Pranav Narayanan Venkit, Tatiana Chakravorti, Vipul Gupta, Heidi Biggs, Mukund Srinath, Koustava Goswami, Sarah Rajtmajer, Shomir Wilson
| Challenge: | 103 peer-reviewed publications on hallucination in large language models (LLMs) are characterized by a lack of agreement with the term ‘hallucination’ in the field of NLP. |
| Approach: | They examine 103 peer-reviewed publications on hallucination in large language models (LLMs) and conduct a survey with 171 practitioners from the field of NLP and AI to capture varying perspectives on halllucination. |
| Outcome: | The findings highlight the need for explicit definitions and frameworks outlining hallucination within NLP and highlight potential challenges. |
Can Third Parties Read Our Emotions? (2025.acl-long)
Copied to clipboard
Jiayi Li, Yingfan Zhou, Pranav Narayanan Venkit, Halima Binte Islam, Sneha Arya, Shomir Wilson, Sarah Rajtmajer
| Challenge: | Existing approaches to infer author’s private states from written text have relied heavily on datasets annotated by third-party annotators. |
| Approach: | They propose a framework for evaluating the limitations of third-party annotations and call for refined annotation practices to accurately represent and model authors’ private states. |
| Outcome: | The proposed methods outperform human annotators on emotion recognition tasks. |
Automated Detection and Analysis of Data Practices Using A Real-World Corpus (2024.findings-acl)
Copied to clipboard
| Challenge: | a crowd-sourced annotation tool matches data practices with policy excerpts . the complexity of privacy policies often deter users from reading them . |
| Approach: | They propose an automated approach to identify and visualize data practices within privacy policies at different levels of detail. |
| Outcome: | The proposed approach matches data practices with policy excerpts at different levels of detail. |
Nationality Bias in Text Generation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that nationality biases in language models can be a factor in improving the performance of social NLP models. |
| Approach: | They propose to use a text generation model, GPT-2, to analyze how the number of internet users and the country’s economic status affects the sentiment of stories. |
| Outcome: | The proposed model accentuates biases about country-based demonyms and reduces them with the use of adversarial triggering. |